93 


Malaysian Journal of Learning and Instruction: Vol. 8 (2011): 93-115 

Teachers’ Knowledge and Understanding of the 
Malaysian School-Based Oral English Assessment 

GURNAM KAUR SIDHU 
CHAN YUEN FOOK 1 

Faculty of Education 
Universiti Teknologi Mara 
yuenfook@hotmail. com 1 

AZLEENA MOHAMAD 

SMK Dato ’ Harun 
Tanjung Karang, Selangor 

ABSTRACT 

Purpose - The paper sought to investigate TESL teachers ’ knowledge 
and understanding of the Malaysian School Based Oral English 
Assessment (SBOEA) after five years into its implementation in 
upper secondary ESL classrooms in Malaysian public schools 

Method - The descriptive study involved a total of 80 TESL trained 
teachers from the 19 schools located in a district in Selangor, 
Malaysia. The study employed a three-prong strategy to collect 
data which included a questionnaire, semi structured interviews and 
document analysis. 

Findings - The findings of the study indicated that there was a low 
compliance (58.7%) rate among teachers to the procedures laid 
out in the SBOEA Manual with regards to the implementation of 
SBOEA. The teachers also acknowledged their lack of knowledge 
and understanding of the SBOEA and a half of the teachers in the 
study admitted having problems with the scoring criteria. Some 
teachers highlighted that they felt the SBOEA lacked reliability and 
validity within and outside the schools. 

Significance - The paper explores the possible implications of the 
study to the enhancement of the implementation of the School Based 
Oral English Assessment in Malaysian public schools. 
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INTRODUCTION 


Assessment is often seen as an integral part of the teaching and 
learning equation as it helps both teachers and parents alike to 
monitor the progress and performance of a student. Researchers 
such as Dietel, Herman and Knuth (1991) and Borich and Tombari 
(2004) posit the view that for real learning to take place, effective 
assessment and evaluation mechanisms have to be put in place and 
therefore it is pertinent for all stakeholders involved in the business 
of education to possess sound knowledge and understanding of 
current assessment and evaluation mechanisms. Traditionally, 
sessments were summative and centralized and this often 
resulted in teachers placing a lot of emphasis on examinations. 
Recent findings (Cunningham, 1998; Dietel, Herman & Knuth, 
^—1=991; Borich & Tombari, 2004) have however, argued that the 
one standardized test meted out in summative evaluations is not 
an accurate indicator of a student’s actual performance or ability. 
Today, assessment policies all around the globe have undergone 
a gradual but significant paradigm shift from summative to more 
formative based assessments. Boud and Falchikov (2005) make 
the salient point that formative assessment is assessment at regular 
intervals of a student’s progress with accompanying feedback in 
forder to help to improve the student’s performance. 


■"4*1 keeping with global trends of testing and evaluation, both 
“policymakers and educators in Malaysia too are now looking towards 
^•■school based assessment (SBA) as a catalyst for educational reform, 
it is seen as a form of leverage for instructional improvement to assist 
teachers to find out what students are learning in the classroom and 
how well they are learning it. Such an opinion was first articulated by 
the then Malaysian Minister of Education, Tan Sri Musa Mohamed 
when he said that: 


“we need a fresh and new philosophy in our approach 
to exams ... we want to make the education system 
less exam-oriented and (we) are looking at increasing 
school based assessment as it would be a better gauge 
of students’ abilities.” 


(New Straits Times : 2003, May 8, p.7). 
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In line with the changing trends in assessment, school-based 
assessment or commonly referred to as PKBS (Penilaian Kendalian 
Berasaskan Sekolah) in Malaysia was introduced into Malaysian 
schools under the New Integrated Curriculum for Secondary Schools 
(KBSM). Since the late 1990s it has adopted ‘coursework’ for a few 
subjects in secondary schools such as History, Geography, Living 
Skills and Islamic Education for the lower secondary classes and 
Biology, Chemistry and Physics for the upper secondary classes. 
Later in 2002 the School Based Oral Assessment (SBOA) was 
mplemented for both Bahasa Malaysia (the national language) 
and English Language for Secondary Five students sitting for the 
2003 SPM Examination (equivalent to the UK GCE ‘O’ Levels 
Examination). It was against this background that the School Based 
Oral English Assessment (SBOEA - hereafter) was first implemented 
in 2002. More importantly it was viewed as a step forward 
towards formative on-going assessment for enhancing student 
learning, specifically students’ speaking/communicative skills. The 
implementation of the SBOEA was outlined in a comprehensive 
document called the SBOEA Manual or Pengendalian Ujian Lisan 
Berasaskan Sekolah (ULBS) which was published and dsitributed to 
schools in 2002. 

The rationale of the SBOEA as spelt out in the manual 
highlighted that the SBOEA is an authentic form of assessment 
that could assess the student’s actual speaking activity. It also 
promotes the teaching and learning process in the ESL classroom 
*^as it focuses on ongoing oral assessment from Secondary Four to 
Secondary Five. Furthermore it is in line with the requirements of 
the Curriculum Specifications enabling more than one type of oral 
skill to be assessed. More importantly, the classroom teacher will be 
the assessor. In such a situation the classroom teacher is the person 
most familiar with the student and is therefore better informed of the 
student’s actual abilities (Ministry of Education, 2002). 

Under the SBOEA, the oral assessment is integrated in the 
everyday teaching and learning process. Students are assessed a 
total of three times by their classroom ESL teacher over the two 
years based on four models of oral assessment. In Secondary Four 
students are required to do Model 1 (individual presentation) and 
Model 2 (individual presentation with prompts) whilst in Secondary 
Five they have a choice between Model 3 (pair work) or Model 4 
(group work). The students are graded based on 5 constructs and 6 
scoring bands (Excellent = 6, Good=4-5, Satisfactory=3, Weak=2, 
Very Wcak= 1) with a maximum score of 30 marks (6 bands x 5 
constructs). 
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It is often said that ‘change’ is the only constant in the workplace. 
Rae (2006) notes that innovation is inevitable and it is only a matter 
of time before classroom teachers would have to adopt and accept 
the way forward. Therefore with the SBOEA making its way into 
the Malaysian ESL classrooms, the rate of adoption and reaction 
by teachers may vary considerable. According to Rogers (2003) in 
his theory of diffusion, there are five categories of product adopters. 
While some teachers like the ‘innovators’ may embrace this change 
willingly with ease, the ‘early adopters’ and the ‘early majority’ 
wlfo may take this new form of assessment when more information 
is available and they can see and experience the benefits. Others like 
iaihe sceptical ‘late majority’ may take more time to bring change 
,_into their classrooms. Meanwhile the ‘laggards’ may resist or stall 
the change process as they often find change difficult. This group 
often finds it difficult to move away from their established and 
traditional ‘comfort zones’. Hence it is important to embark on a 
study to investigate the effectiveness of SBOEA and investigate at 
what point of adoption these teachers are, so that follow up steps can 
be taken to ensure the success of new endeavours like the SBOEA. 

To measure the effectiveness of a new formative assessment 
^programme like the SBOEA, it is pertinent to examine how teachers 
view SBOEA and how it is implemented in the ESL classroom. 
Therefore, this study sought to investigate TESL teachers’ 
perspectives on the implementation of the SBOEA. More specifically 
the study looked into teachers’ knowledge and understanding of 
the implementation of SBOEA at the upper secondary levels; i.e. 
Secondary Four and Secondary Five levels. 


d 


sessing Speaking 


In most Malaysian ESL classrooms, the assessment of speaking is 
often a neglected activity with teachers putting more emphasis on 
reading and writing skills because they are often the skills tested 
in public examinations. This has also been noted by Chaudhary 
(2008) who highlights that the teaching and testing of reading and 
writing skills are more established than speaking in the ESL/EFL 
classes because of the absence of an accepted list of items to be 
taught and tested and logistics issues such as time and venues for 
testing. Furthermore, written examinations can be administered 
to hundreds of candidates at the same time, while speaking tests 
have to be administered to examinees individually or in small 
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groups, making the whole process expensive and time consuming. 
Chaudhary (2008, p.3) reiterates that as candidates’ responses are 
oral, hundreds of candidates speaking can reduce the ESL/EFL 
classroom into "cacophony of unassessable noise". 

One of the key challenges in testing speaking is to design 
tasks that elicit spoken language of the type and quantity that 
will allow meaningful inferences to be drawn from scores to the 
candidate’s ability on the construct the test is designed to measure 
(Fulcher, 2003). When assessing speaking, Hartley and Sporing 
(1999) stress that language assessors are usually interested in the 
candidate’s ability to perform in a range of situations much wider 
than those that could be sampled during the test. In order to do this, 
Messick (1989) brought to light two threats the test must avoid to its 
construct validity. The two threats are construct under-representation 
and construct-irrelevant variance. Construct under-representation 
refers to the extent to which a test fails to capture important aspects 
of the construct the test is intended to measure, while construct- 

E irrelevant variance is the extent to which test scores are influenced 
by factors that are irrelevant to the construct that the test is intended 
to measure. The SBOEA has to some extent been able to address this 
shortcoming because formative classroom assessment is continuous 
and on-going with speaking tasks integrated in everyday ESE 
■ lessons. 

i 

Luoma (2004) emphasizes that assessing speaking requires 

E teachers to first define the kind of speaking they want to test before 
proceeding to develop test tasks and rating criteria. Furthermore, 
Fulcher (2003) notes that the fundamental questions that one need 
to ask when preparing speaking tasks are: “will the task elicit a 
performance that can be scored? and “will it be possible to make 
inferences from the score to the construct we intend to measure?” 
Furthermore, Weir (1993) reiterates that speaking tasks chosen 
should also be the most efficient way of obtaining the information 
about the test-takers. On the subject of test tasks and rating criteria, 
Fuoma (2004) draws attention to the fact that when preparing 
scoring criteria thought should also be given to the aspect of 
inter-rater reliability as teachers need to understand the difference 
between subjective or objective judgment in various purposes of 
the speaking tests. Arguably, steps must also be taken to ensure 
credible procedures are established in the test design and rating 
score whether the final assessment score calls for different raters or 
just one teacher. Fuoma (2004) contends that this consideration is 
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often neglected in the ESL/EFL classroom assessment procedures as 
teachers depend heavily on textbooks and materials that sometimes 
do not match the speaking tests meted out to the students 

Bachman (1990), Weir (1993) and Louma (2004) highlight 
that for ESL teachers to conduct effective oral assessments, they 
need to possess cogent knowledge about applied linguistic theories 
and valid testing procedures so that they can map out the direction of 
their assessment strategy. Kim (2003) also maintains that assessment 
needs to be theory-driven as concepts such as validity, reliability and 
efficiency affect assessment. Bachman (1990) further reiterates that 
a valid and reliable test is useless if it is not practical. Practicality 
involves the questions of economy, ease of administration, scoring 
and interpretation of results. Weir (1993) posits the view that 
Tsfassroom tests should not require expensive specialized equipment 
Qjj) highly trained examiners or raters. He adds that though classroom 
teachers are often pressured to ensure speaking tests are short and 
practical since they do not have the liberty of time in assessing 
students’ communicative ability, this should never be allowed to 
affect test validity. 

Adding to this discourse, Fulcher (2003) further draws 
attention to the fact that assessing speaking involves more than just 

_lo oking at the type of speaking test and task and other theory driven 

concepts like validity, reliability, practicability and authenticity. He 
—aotes that there are often a variety of other factors that teachers need 
to consider such as the learner variables. There is consensus among 
researchers such as Derwing and Rossiter (2003) and Dalton anf 
Seidlhofer, (1994) that learner factors such as motivation, attitude, 
“^aptitude, LI background, social status, interests and talent all have 
been well researched and have proven to contribute to the success 
Itafiifailure in the learner performance in speaking tests and in the 
acquisition of effective speaking skills. For instance, Fulcher and 
Reiter (2003) investigated on task difficulty in speaking tests and 
found that extreme conditions of social powers or imposition may 
provide an indication of greater task difficulty for some test-takers. 
Moere (2006), on the other hand, looked at the validity evidence in 
a university group oral test. His study showed that the test scores 
were reliable and the greatest systematic variation in test scores is 
contributed by the person-by-occasion interaction. The study also 
revealed that candidates’ performances or how raters perceive 
a candidate’s ability could be affected by the characteristics of 
interlocutors and interaction dynamics within the group. 
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Breeze (2004) observes that the testing of speaking is still a relative 
‘new field’ within the contexts of applied linguistics and there still 
remains to date a number of issues that require empirical evidence. 
Therefore it was against this background of scant empirical research 
on assessing speaking that this study sought to investigate a school- 
based formative speaking test which has been recently introduced 
into the Malaysian ESL classroom- commonly referred to as the 
School Based Oral English Assessment (SBOEA). 


The Study 

The study employed a descriptive study design to identify the 
perception of teachers towards the implementation of School 
Based Oral English Assessment (SBOEA) in Malaysian secondary 
schools. A descriptive study was chosen to allow a qualitative and 
quantitative description of the relevant features of the data collected 
(Cohen, Manion & Morrison, 2003). More specifically the study 
sought to investigate teachers’ knowledge and understanding of the 
implementation of the SBOEA with regards to the following: 

• objectives of the SBOEA, 

• the implementation of the SBOEA 
pre administration of the SBOEA 

• administration of the SBOEA 

• post administration of the SBOEA 

• challenges teachers faced 

• suggestions for the enhancement of SBOEA 

Population Sample 

The study involved all the 19 (18 public and 1 private) secondary 
schools located in a district in the state of Selangor, Malaysia. The 
sample comprised all 89 TESL trained teachers teaching English 
at the upper secondary levels, i.e., Secondary Four and Secondary 
Five. The total population was included as educational researchers 
(Cohen, Manion & Morrison, 2003; Parmjit, Chan & Sidhu, 2006) 
stress that for a small population of 100 or less, it is best to survey 
the entire population. 
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Instrumentation 


The research methodology employed a three-pronged approach 
which involved the use of a questionnaire, semi structured 
interviews and document analysis. Since it involved a total of 19 
schools, a total of nineteen reports from the 19 Chief Assessors 
were analyzed for the document analyses. For the interviews, two 
teachers were randomly selected from each of the four (4) types of 
schools in district - i.e. fully residential school, daily residential 
school, religious school and day-time daily school. The interview 

£ sessions, which involved eight teachers, asked a variety of questions 
on the implementation of the SBOEA in their respective schools 
(refer to the Appendix). The SBOEA Chief Assessors Reports were 
based on the sample provided by the Ministry of Education. These 
. reports looked into a number of the components with a focus on 
Vfche compliance of schools to the guidelines outlined in the SBOEA 
Manual. The findings helped to triangulate findings obtained form 
both the questionnaire and the interviews. 

The questionnaire used was adapted from a similar study 
conducted of Chan, Sidhu & Mohd. Rizal (2006). The questionnaire 
“consisted of both close and open-ended questions that were used 
to obtain data both in quantitative and qualitative forms. The 
^.ad apted questionnaire was validated by two experienced university 
lecturers to ensure the construct validity of the instrument. A pilot 
_t@st was carried out in a secondary school in Selangor and the 
ridliability measure for the instrument reported a Cronbach Alpha 
of 0.80. Quantitative data was analysed by using frequency and 
^percentage. For certain key areas, mean was calculated based on the 
percent indicated. Besides that, the qualitative data was analysed 
both inductively and deductively to identify the main themes that 
emerged based on the research questions posed in this study. 

The questionnaire used in this study consisted of 4 parts. Part 
1 sought information on the demographic profile of the respondents 
and included aspects such as gender, age, ethnicity, teacher training 
qualification, academic qualification, current position and working 
experience. Apart from that, this part of the questionnaire also 
elicited information relating to SBOEA which included the exposure 
to school based assessment, courses on SBOEA and details about its 
organizer (if any). Part 2 of the questionnaire focused on obtaining 
information on the respondents’ knowledge and understanding 
of the SBOEA. Part 3 focused on the three following aspects in 
the SBOEA Manual: the preparation for SBOEA, administering 
SBOEA and scoring. Finally, Part 4 investigated the implementation 
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and feedback aspects of SBOEA. This part of the questionnaire 
included both open and close ended questions. Most of the other 
sections required respondents to provide their responses to given 
items/statements based on a 4-point Likert scale where a score of 
1 indicated strong agreement whilst a score of 4 indicated strong 
disagreement. 


FINDINGS 

The following section puts forward the main findings obtained from 
the survey instrument. Where necessary data from the interview 
sessions and the Assessors’ reports will be added to further 
triangulate the findings presented in this paper. The survey achieved 
a 89.9 per cent response rate as 80 out of the total population of 89 
Secondary Four and Five TESL teachers responded to the study. 
Discussion of the findings begins with the respondents’ demographic 
profile, knowledge and understanding of SBOEA, challenges in 
implementing SBOEA and ways of enhancing the implementation 
of the assessment. 

Respondents ’ Demographic Profile 

Out of the total 80 teachers, 60% were females and another 40% 
were males. Approximately 18.8% of the respondents were Diploma 
holders while the majority of them (69.9 %) were degree holders 
and 11.3% possessed a Masters. Even though close to 90% of the 
respondents indicated that they had no official training for the 
SBOEA, all claimed they were qualified SBOEA assessors as they 
had been appointed by their school principals. Under the SBOEA, 
an assessor must posses a letter of appointment to be able to become 
an assessor. Therefore, most schools often provide a letter of 
appointment to TESL teachers if they are teaching Secondary Four 
and Secondary Five ESL classrooms. With no formal training they 
would without doubt have issues relating to the implementation of 
and scoring for the SBOEA. 


Knowledge and Understanding of School-Based Oral English 
Assessment 

One of the main aspects investigated in this study was the respondents’ 
basic knowledge and understanding of the implementation of 
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SBOEA as spelled out in the SBOEA Manual. We explored teachers’ 
knowledge and understanding on various SBOEA aspects such as the 
assessment objectives, implementation, preparation, administration 
and scoring. For effective implementation of SBOEA it is important 
to ensure 100% compliance, i.e., adhered to by all teachers. The 
findings exhibited in Table 1 however revealed that the teachers 
possessed rather limited knowledge and understanding as most of the 
scores registered were below the 80% compliance rate as postulated 
the Ministry of Education (MOE) during their training sessions. 



Bleachers’ Knowledge and Understanding of the SBOEA 


Area 

0 

■ 

Item 

No. 

Item 

Score/ 
Percent of 
Correct 
Answer 

Mean 

CD 

t 

CD 

1 

SBOEA develops a candidate’s 
ability in line with the learning 
objectives as stated in the English 
Language Syllabus for Malaysian 
Secondary Schools. 

56/80 

70.0% 


—© 

> 

c 

m 4^ 

C/3 

2 

SBOEA aims to assess candidates’ 
ability to use English in everyday 
life. 

62/80 

77.5% 

63.3% 

0 

3 

SBOEA has eight testing 
objectives. 

34/80 

42.5% 



ii 

=sEh.2 

—E 

4 

5 

SBOEA offers four models of oral 
assessment. 

Candidates are assessed twice 
in Secondary Four and once in 
Secondary Five. 

36/80 

45.0% 

36/80 

45.0% 

45.0 % 

1 ^ 

CD C 
£.2 
c 2 

6 

Candidates are allowed to choose 
their own activities from the list of 
activities provided in the SBOEA 
manual. 

64/80 

80.0% 


•2 22 
cd’3 
h’ d 

gS 

7 

Candidates decide on the topic and 
task for the assessment. 

48/80 

60.0% 

65.8% 

%< 

CU 

8 

Candidates prepare their task with 
teacher’s consultation. 

46/80 

57.5% 



(Continued) 
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Area Item 
No. 


Item 


Score / Mean 
Percent of 
Correct 
Answer 


9 

SBOEA is conducted during the 
teaching and learning process. 

52/80 

65.0% 

10 

Candidates are allowed to make 
reference to the text during 
assessment. 

38/80 

47.5% 

11 

Each candidate is given a time limit 
to speak. 

52/80 

65.0% 

12 

Each candidate is allowed to do 
the assessment of each model more 
than once. 

42/80 

52.5% 



13 

Candidates are assessed based on 

50/80 



five constructs. 

62.5% 


14 

Marks are awarded in five bands: 

64/80 



Excellent, Good, Satisfactory, 

80.0% 

'S' 


Weak and Very Weak. 


.o 

■D 

15 

Each band is provided a list of cri- 

58/80 

*-< 

cn 


teria/ descriptor. 

72.5% 

C 

’2 

16 

The highest score for each band is 

48/80 



6. 

60.0% 

<3 



62.1% 

C A 

17 

The grand total for SBOEA is 30 

58/80 

Dh 


marks. 

72.5% 

W) 

_C 

18 

The highest score is recorded as the 

40/80 

5-h 

o 

o 


final score. 

50.0% 


19 

Scores are recorded in the Oral 

80/80 



English Test (O.E.T) Individual 
Profile form. 

100.0% 


20 

Candidates sign the form after each 

34/80 



assessment. 

42.5% 


Overall Mean 


58.7% 


Note: The mean was obtained from the average score achieved by teacher 
respondents on each item 


Objectives of SBOEA 


The SBOEA Manual has identified eight testing objectives: 
converse on a topic effectively, speak fluently, speak coherently, 
give appropriate responses, speak using the language appropriately 
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within context, speak using correct and acceptable pronunciation, 
speak using correct grammar and speak using a wide range of 
appropriate vocabulary (MOE, 2002). The findings showed that 
even though a majority of the respondents were aware of the 
general and specific objectives of the SBOEA, only less than half 
(45%) were aware of its main testing objectives. This is rather 
disappointing as these objectives are of paramount importance since 
the testing objectives include the assessment constructs which every 
SBOEA Assessor must be aware of. Teachers must be aware of 
the objectives to ensure they help students to achieve the learning 
outcomes postulated by the SBOEA. 



Implementation of SBOEA 

ir this section, respondents were asked about the models of SBOEA 
d the frequency of oral assessment conducted in their classroom. 
With regards to the implementation, respondents indicated limited 
knowledge of the SBOEA as only 45 % of the respondents were 
Svarc that there were four models of oral assessment, and that 
candidates were assessed twice in Secondary Four and once in 
Secondary Five. This could probably be due to the fact that not all 
the respondents may be teaching at both the upper secondary levels. 

In the administration of SBOEA, the results were divided 
three phases: ‘pre-administration’, ‘while administration’ and 
ost administration’. The pre-administration phase involved the 
eparations respondents made prior to the actual administration 
!q| SBOEA. The manual states that candidates of SBOEA need to 
consult with their teacher in deciding on the type of activity, choice 
of topic and task for the intended assessment. In this oral assessment, 
teachers fulfilled the role of assessor, consultant, prompter and 
cilitator. 

Respondents had better knowledge and understanding on the 
preparation of the SBOEA as a majority (80%) of them indicated 
awareness that candidates were allowed to choose their own 
activities and can decide on the topic and task for the assessment. 
However, only slightly more than half of the respondents (57.5%) 
knew that school candidates need to prepare their task with teacher’s 
consultation. 

The ‘administration’ phase highlighted the procedures 
involved while conducting the oral assessment. Under the stipulated 
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0 



Qs 


regulations the assessment is to be conducted during the teaching and 
learning process; however only 65% of teacher respondents were 
aware of this. Furthermore, slightly more than half (52.5%) were 
aware of the fact that candidates were allowed to do the assessment 
of each model more than once, and less than half (47.5%) realized 
that candidates were actually allowed to make reference to the 
text during assessment. Besides that, only 65% of them knew that 
there was no time duration for student presentation as stipulated 
in the SBOEA Manual. Findings at this phase implied that teacher 
espondents were not completely aware of the procedures involved 
in conducting the oral assessment thus, may not conform to the 
intended practice. 

According to the SBOEA Manual, the post administration 
phase highlighted the scoring procedures of SBOEA. The scoring 
criteria had changed three times in two years, moving from eight 
constructs to five constructs. Some of the respondents were rather 
confused about the scoring criteria. Therefore a lack of effective 
compliance was also seen in the post administration especially in 
the scoring (62.1%) of the SBOEA. The findings further exhibited 
that only 62.5% of the respondents were aware that the SBOEA 
was at present assessed based on five and not eight constructs. 
This was a matter of concern as these teachers were already in the 
process of assessing students’ speaking skills. A majority (80%) 
of the respondents indicated an awareness of the five performance 
bands used in the scoring scheme, but only 75% acknowledged that 
descriptors were provided for each band. Furthermore only 60% 
were conscious of the fact that the maximum score was 6, as the 
previous scoring system used a maximum score of 10. Slightly 
more teacher respondents were aware that the grand total for the 
BOEA was now 30 marks. It was also a matter of grave concern 
that only 50% of the respondents were aware that the highest score 
is to be recorded as the students’ final score, raising the question of 
whether teachers were indeed recording accurately students’ scores 
for the SBOEA. However, respondents revealed 100% compliance 
to the recording to the scores and all of them knew that scores are 
to be recorded in the Oral English Test (O.E.T) Individual Profile 
form. Nevertheless less than half (42.5%) were conscious of the fact 
that students must sign the form after each assessment. This showed 
that some of the respondents did not adhere to the given guidelines 
as there is a column provided for the candidate’s signature in the 
assessment form. 
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Challenges in Implementing SBOEA 


This study also investigated the major challenges faced by 
teachers implementing SBOEA in their ESL classroom. Based on 
data obtained from the open ended questions posed in the survey 
instrument (Part 4. Item 4), the interview sessions and the 19 
Assessors’ reports, the findings (Table 2) can be summarized into 
three main categories namely, the teacher factor, the student factor 
and the educational system. 


Table 2 

Challenges Identified by TESL Teachers 



Teachers’ challenges in 
implementing SBOEA 


Percent/ Frequency N = 80 


Strongly Agree Disagree Strongly 
Agree Disagree 


Teacher Factor 


tack of knowledge / 

10% 

32.5% 

45% 

12.5% 

80 

understanding of SBOEA 

(8) 

(26) 

(36) 

(10) 


"Lack knowledge in grading 

20% 

30% 

37.5% 

12.5% 

80 


(16) 

(24) 

(30) 

(10) 


Difficulty understanding 

18.4% 

34.2% 

39.4% 

7.9% 

76 

-Band descriptors 

E= 3J93£ue and unclear) 

(14) 

(26) 

(30) 

(6) 


Heavy workload 

52.5% 

(42) 

22.5% 

(18) 

25% 

(20) 


80 

''“■“•■Student Factor 

Too many students/ 
candidates to be assessed 

45% 

(36) 

40% 

(32) 

15% 

(12) 


80 

Students’ discipline 

44.7% 

36.8% 

10.5% 

7.9% 

76 

* 

(34) 

(28) 

(8) 

(6) 


—^■System Factor 

Time constraint 

57.5% 

(46) 

25% 

(20) 

17.5% 

(14) 


80 

Lack of resource materials and 

15% 

27.5% 

45% 

12.5% 

80 

references 

(12) 

(22) 

(36) 

(10) 


Lack of support from school 

15% 

20% 

40% 

25% 

80 

administration 

(12) 

(16) 

(32) 

(20) 


Too focused on central exam 

26.3% 

(20) 

42.1% 

(32) 

31.6% 

(24) 


76 
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The findings presented in Table 2 highlight that teachers face seven 
(7) major challenges in conducting the SBOEA. The challenges are 
as follows: lack of knowledge/understanding of SBOEA, lack of 
knowledge in giving scores, heavy workload, too many students/ 
candidates to be assessed, students’ discipline, time constraint, and 
schools been too exam-oriented. 

The results indicated that close to 42.5% of the respondents 
claimed lack of knowledge and/or understanding of the assessment 
as a challenge. This implies that knowledge may impede teachers’ 
ability to conduct the SBOEA effectively in the ESL classrooms. 
Furthermore, the findings also revealed that 50% of the respondents 
agreed that they lacked knowledge in grading and another 52.6% 
felt they had some difficulty understanding the band descriptors as 
they were unclear. 

The band descriptors were also identified as one of the 
challenges from the interview. An informant claimed that it was 
difficult for him to differentiate between ‘excellent’ and ‘good’. He 
added that this was because he made comparison with the students 
in his school. Thus, he said what he claimed to be excellent, may 
pass as satisfactory in another school. This claim was similar to one 
made by another interview informant. She claimed that she tended 
to compare her students with the weakest class. Hence, at times, 
she marked her students high when she saw them doing better than 
the weaker ones. This showed that teacher respondents did not 
understand the descriptors, thus affecting the scoring of SBOEA. 
When the guideline was vague, it was difficult for the assessors to 
interpret and award scores to their candidates. 

A total of 60 respondents identified heavy workload as a 
challenge to assess student’s communicative ability. 42 teacher 
respondents strongly agreed that this was a challenge to them 
while 18 stated that they agreed. This showed that workload was 
an obstacle for the respondents in their effort to implement the 
assessment in their ESL classroom. Heavy workload was ranked 
the second highest among the challenges identified. The number 
of candidates to be assessed was also seen as a challenge: 45% of 
the respondents strongly agreed that it was difficult to assess a large 
number of students. It was seen as one of the major challenges as the 
percentage was high on ‘strongly agree’ option compared to ‘agree’ 
option. This challenge was closely related to the large classroom size 
in their schools. An interview informant commented that she had 
three (3) classes of Secondary Four with an average of 45 students 
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in each class. She complained that it was tiring and unpractical to 
assess about 135 candidates twice in a school year. Hence, having 
a large number of students did influence the implementation of 
SBOEA in the classroom. 

Student discipline was also seen as one of the major challenges 
to SBOEA. 34 respondents strongly agreed that it was one of the 
constraints they faced in implementing SBOEA, while 28 opted for 
‘agree’. This claim was supported by the responses obtained in the 
heads of assessor report. Analysis revealed that more than half of 
them identified students’ seriousness and preparation as a constraint 
in conducting SBOEA. Evidence from interview informants also 
showed that students were not serious about SBOEA, thus delayed 
the task of assessing them. It can be concluded that the problem of 
oral assessment was not only time consuming but also stressful for 
CJB e teacher. 

Time was also seen as a challenge in implementing SBOEA. 
Tjiis was evident as 66 respondents identified this as one of the major 
l^ehallcngcs. A total of 57.5% of respondents selected the ‘strongly 
agree’ option, indicating that this was a major constraint to them. 
This claim was supported by the data gathered from the document 
analysis, which revealed that 13 out of 19 heads of school assessors 
—id entified time as a constraint in their report. One interview informant 
she stressed that assessing 135 students was time consuming. At the 
same time, she did not have time to do it in class that she had to do 
in isolation. This showed that time constraint has led the teacher 
violate the requirement of conducting the assessment during the 
eaching and learning process. 

68.4 % of the respondents also felt that school was too 
exam-oriented, and this posed a challenge to SBOEA. During 
the interview session these teachers highlighted that students, 
parents and schools (especially top school management) paid more 
attention to centralized public examinations compared to school 
based assessments. The results of the public examinations were 
taken seriously by both schools and parents as a measure of school 
accountability and individual pride (Lim & Zhao, 2005). This has 
led to a change in the teaching practice in schools - teaching to the 
test, thus setting a rather depressing exam-oriented culture among 
students, teachers, parents and administrators alike (Yunus, 2001). 
Teachers tended to focus on the content of the subject syllabus 
forecasted for the examination, abandoning other aspects of the 
syllabus that may be useful for a holistic development. 




http://mili.uum.edu.rn 
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Enhancing the implementation of SBOEA 

The study also sought respondents’ views on to how SBOEA could 
be further enhanced. The findings are displayed in Table 3. Data was 
gathered from the survey (Part 4, Item 5) administered during the 
study and the interview responses. 70% of the respondents in this 
study highlighted that teachers should be provided with samples of 
students’ performance for each band. This suggestion was probably 
seen as a solution to the current bands descriptors which were 
perceived ‘vague and unclear’. It also indicated that respondents 
were concerned with the scoring of SBOEA. Teachers pointed out 
in the interview sessions that if sample scripts were presented for 
each band, the reliability of scoring within classes and schools could 
be further established. 

Table 3 

Enhancement of the SBOEA (n = 80) 


Suggestions 

Frequency 

Percentage 

(%) 

Provide samples for each band descriptors 

56 

70.0 

Reduce the number of assessments 

52 

65.0 

Specify teacher’s role in the consultation process 

50 

62.5 

Send assessors for SBOEA courses 

48 

60.0 

Provide a variety of activities 

36 

45.0 

Specify SBOEA’s time frame 

22 

27.5 


52 of the respondents (65%) suggested that the number of assessments 
be reduced. Currently there are two assessments for Form Four and 
one for Form Five. Interview data indicated that teachers felt that the 
SBOEA involved “too much paper work”; large class enrolments 
did not help. A majority saw reducing the number of assignments as 
the ‘best solution’ as they had to deal with having a large number of 
candidates to assess. They also believed that it could provide them 
with more time to conduct the assessment. 

The findings also revealed that respondents would like the 
teacher’s role in the consultation process to be further clarified. This 
was because they did not want the candidates to depend too much 
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on the teachers for assistance. Findings from interview sessions 
further highlighted that there was concern about how much help 
teachers should give if the final score went towards the public SPM 
examination. There was concern that some schools gave ready made 
scripts to their students to memorize whilst others offered little or 
no help. 

Respondents also suggested the need for more training and 
exposure to SBOEA. 60% of the respondents felt that they needed 
to attend courses on the SBOEA. The demographic profile indicated 
s thift 90% of the respondents had not attended any course on SBOEA. 
Thus, this could have been the reason for the recommendation, 
although it ranked fourth place in the list. 

Since the SBOEA has to be conducted during the English 
~ Jianguage lessons, teachers were concerned about implementation, 
tllosc to 45% of the respondents saw the need to provide teachers 
with samples of a variety of activities that could be done in the ESL 
elassroom. Currently the manual has no such examples. 

Interview sessions also revealed that teachers were concerned 
about when SBOEA must be conducted. 28% said that they would 
appreciate a specific time frame so that all schools would conduct 
the SBOEA during the same period. This would enhance the 
“Coordination of school, regional or district meetings. 

Besides the above enhancement strategies, a number of 
teachers also pointed out that the ministry had to be more effective in 
disseminating information. Some respondents felt that information 
was slow to reach the rural schools. There were instances when 
Ob ey realized that they were doing the wrong thing as information 
did not reach them in time. Some respondents recommended 
that the Malaysian Examination Syndicate should opt for a more 
/“technology-savvy idea such as having an e-portal for the SBOEA. 
A few teachers also drew attention to the type of training model 
„ .used. They felt that the current Cascade Training Model used by the 
f°Ministry of Education was ineffective. They suggested that more 
in-house and centralized training workshops be held to increase 
teachers’ knowledge and understanding of the SBOEA. 


IMPLICATIONS AND RECOMMENDATIONS 

This study involved only 80 TESL teachers from 19 secondary 
schools in a district located in the state of Selangor. Due to the 
small sample size, the findings cannot be generalized to the total 
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population of secondary TESL teachers in Malaysia. Nevertheless 
a number of pertinent issues and concerns can be drawn from the 
study with regards to the implementation of the SBOEA, which 
would benefit upper secondary Malaysian teachers in general. 

It is perhaps important to note that effective implementation 
of any programme demands a 100% compliance rate to stipulated 
guidelines. This study however has indicated that teachers’ 
knowledge and understanding of the implementation of the SBOEA 
at the 19 schools left much to be desired. All aspects of the SBOEA 
implementation - i.e. Purpose, Implementation Pre Administration, 
During Administration, Post Administration were well below the 
mastery level of 80%. The lack of compliance probably reflects 
a lack of effective training and monitoring procedures. School 
management and education personnel at district, state or national 
education departments perhaps need to be more aware of this and 
take appropriate action. 

Another important concern is teacher competency. Teachers 
admitted to a lack of knowledge and understanding of the SBOEA 
implementation and further expressed their concerns about grading 
students. Teachers’ failure to comply with SBOEA guidelines 
and their lack of understanding and knowledge clearly is a cause 
for concern. The relevant authorities need to address this issue as 
close to 90% of the respondents in this study have had no formal 
training and exposure to SBOEA. More effective training modules 
and e-learning portals could be seen as alternatives to the traditional 
cascade training models currently implemented. 

Furthermore frequent changes in scoring criteria coupled 
with vague descriptors and bands suggest a lack of thought the 
SBOEA by the relevant authorities. This has left many teachers 
confused. Furthermore only about three quarters of the teachers 
were knowledgeable about the band descriptors. Such a scenario 
may reduce the reliability of the SBOEA. There is no denying that 
the reliability of speaking tests also rests upon the competency of 
oral examiners or raters. Fulcher (2003) notes that in most speaking 
assessments inter-rater reliability tends to be low, and therefore more 
time should be devoted to examiner training and standardization. 
Opting for a training manual that would provide SBOEA assessors 
a more hands-on-training workshop would ensure theory-based 
concepts of testing such as validity and reliability are established 
(Louma, 2004; Fulcher, 2003; Weir, 1993). 
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Students’ commitment, proficiency and discipline are also issues 
that the TESL teacher needs to come to terms with. A majority of 
students still need help as speaking has been a much neglected skill 
in the Malaysian ESL classroom. Hence, mechanisms need to be 
put in place to help students. Arguably, teachers, with their heavy 
workloads need to be advised as to the amount of help and the form 
of supervision that they need to provide. These aspects need to be 
addressed in the SBOEA Manual. Teacher workload and constraints 
of having to assess many students within a stipulated time frame are 
also bones of contention for the TESL teachers. It is perhaps timely 
for Malaysian ESL classrooms to opt for more current approaches 
to assessment such as those provided in computer-assisted language 
learning (CALL). Researchers such as Pennington (1999), Neri, 
Cucchiarini, Strik and Boves (2002), highlight that CALL has 
provided the assessment of speaking skills newer perspectives 
and directions in the development of tests and actual scoring of 
performance. 


including Thoughts 


The TESL teacher in the Malaysian secondary classroom should 
understand that formative assessments like the SBOEA are the way 
""forward as they are to be a permanent feature in the Malaysian 
school system. On 16 October 2010, the Malaysian Education 
Minister announced that the centralized public examination for the 
lower secondary students (known as PMR) would be abolished and 
-^Igplaced by School Based Assessments (SBA). He also stressed that 
■ riiorc measures would be taken to enhance school based assessments 
I-Tita n. 2010). This was further reiterated by the Education deputy 
director, Datuk Noor Rezan Bapoo Hashim who highlighted that 
SBA would be of ‘acceptable standards’ ensuring that ‘teachers 
A and school heads are accountable for matters pertaining to school 
assessment’. She also added that these would include ‘the quality 
of questions’, the marking scheme and ensuring assessments are 
carried out fairly ‘without favouring or victimizing individual 
students” (Tan, 2010, p. E3). These comments also imply that issues 
similar to those highlighted in this study concerning adherence to 
guidelines, maintaining standards and accountability should be 
addressed by teachers implementing school based assessments in 
their classrooms. 
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At this juncture, it is perhaps pertinent to point out that formative 
types of assessment, if properly implemented in schools would be 
a powerful means of improving students’ learning as it provides 
ongoing feedback to the students (Black & William, 1998 as 
cited in Lo Yiu Chun, 2006). Consequently, Malaysian teachers 
should embrace the SBOEA. While it is still at its infancy and 
despite its initial drawbacks, they should view it positively and 
work collaboratively to overcome these limitations. Teachers 
must understand that formative assessment endeavours such as the 
SBOEA have much more to offer than one test score provided by 
traditional centralized examinations. More importantly, the SBOEA 
is a step in the right direction to enhance speaking skills, which have 
been neglected thus far in Malaysian ESL classrooms. 


REFERENCES 

Bachman, L.F. (1990). Fundamental considerations in language 
testing. Oxford: Oxford University Press. 

Borich, G.D., & Tombari, M.L. (2004). Educational assessment 
for the elementary and middle school classroom. New Jersey: 
Pearson, Merill Prentice Hall. 

Boud, D. and Falchikov, N. (2005). Redesigning assessment for 
learning beyond higher education, In Brew, A. and Asmar, 
C. (Eds.), Research and development in higher education 28. 
Sydney: HERDS A, 34-41. 

Breeze, R. (2004). Book Review: Testing Second Language Speaking 
by Glenn Fulcher. In TESL-EJ Journal. Vol. 8 No. 1 Retrieved 
October 2, 2009, from http://www-writing.berkeley.edu. 

Chan, Y.F., Sidhu, G.K., & Md. Rizal Md. Yunus. (2006). The 
knowledge and best practices of secondary school ESL 
Teachers in school-based assessment. Shah Alam: IRDC, 
UiTM. 

Chaudhary, S. (2008). Testing spoken English for credit within the 
Indian university system. TESL-EJ, December 2008, 12 (3), 
1 - 8 . 

Cohen, L., Manion, L., & Morrison, K. (2003). Research methods in 
education. New York: Routledge/Falmer. 

Cunningham, G.K. (1998). Assessment in the classroom: 
Constructing and interpreting tests. London: The Falmer 
Press. 



114 


Malaysian Journal of Learning and Instruction: Vol. 8 (2011): 93-115 


CD 


,im, 


Dalton, C., & Seidlhofer, S. (1994). Pronunciation. New York: 
Oxford University Press. 

Derwing, T.M., & Rossiter, M.J. (2003). The effects of pronunciation 
instruction on the accuracy, fluency, and complexity of L2 
accented Speech. Applied Language Learning, 13, 1-18. 
Dietel, R.J., Herman, J.L., & Knuth, R. A. (1991). What does 
research say about assessment? Oak Brook: North Central 
Regional Educational Laboratory. Retrieved from http://www. 
ncrel./org. 

Fulcher. G. (2003). Testing second language speaking. Harlow: 
Pearson Longman. 

Fulcher, G. & R. M. Reiter. (2003). Task difficulty in speaking tests. 

Language Testing. 20 (3), 321 - 344. 

Hartley, L. & M. Sporing. (1999). Teaching communicatively: 
Assessing communicatively? Language Learning Journal, 
19, 73 - 79. 

C. S., & Zhao, X. P. (2005). Assessment and examination 
system: A comparative study between Malaysia and Shanghai 

E (China). Paper presented at the The Third East Asia Regional 
Conference on Mathematics Education. Retrieved from http:// 
math.ecnu.edu.cn. 

•'ho Yiu Chun (2006). Practice and challenges of school-based 
formative assessment. Hong Kong: The Hong Kong Institute 
of Education. 

Luoma, S. (2004). Assessing speaking. New York: Cambridge 
University Press. 

lessick, S. (1989). Validity. In Linn, R. L. (Ed) Educational 
measurement. New York: Macmillan/ American Council on 
Education. 

/■Ministry of Education, Malaysia. (2002). Pengendalian Ujian Lisan 
Berasaskan Sekolah (ULBS). Malaysia: KPM. 

Moere, A. V. (2006). Validity evidence in a university group oral 
test. Language Testing, 23 (4), 411 - 440. 

Neri, A.,Cucchiarini,C.,Strik,H.,&Boves,L. (2002). The pedagogy- 
technology interface in computer assisted pronunciation 
training. Computer Assisted Language Learning, 15(5), 441- 
467. 



New Straits Times. (2003, May 8). Malaysian education system to 
be less exam oriented. New Straits Times, p.7. 

Parmjit, S., Y. F. Chan & G. K. Sidhu. (2006). A comprehensive 
guide to writing a research proposal. Selangor: Venton 
Publication. 



115 


Teachers' Knowledge and Understanding of the Malaysian School-Based : 93-115 

Pennington, M.C. (1999). Computer-aided pronunciation pedagogy: 
Promise, limitations, directions. Computer Assisted Language 
Learning, 12(5), 427-440. 

Rae, J. (2006), Bringing innovation to the classroom. Insight. 

Retrieved from http://www.businessweek.com 
Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: 
Free Press. 

Tan, E.L. (2010, October 17). School tests will be fair. The Star - 
Education supplement, p. E3. 

Weir, C. J. (1993). Communicative language testing. London: 
Prentice - Hall. 


APPENDIX: SEMI STRUCTURED INTERVIEWS 

Warm - up questions: 

1. How long have you been teaching in this school? 

2. How long have you been teaching English? 

3. Are you an assessor for School Based Oral English Assessment 
(SBOEA)? 

4. How long have you been an assessor for SBOEA? 

Implementation: 

5. Is SBOEA fully implemented in your school? 

6. In your opinion, is it easy to carry out SBOEA in your 
classroom? 

Reliability: 

7. Do you think SBOEA shows your students’ true ability in 
using the language? 

Challenges: 

8. Do you face any problems when you carry out SBOEA in 
your classroom? 

9. What are the problems you face? 

Suggestions to further enhance SBOEA: 

10. What suggestions do you have to further enhance the 
implementation of SBOEA? 



